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DETAILED ACTION 

Response to Amendment 

1 . This office action is in reply to Applicant's Response dated August 1 7, 201 1 . 

2. Claims 1, 5, 8, 12, 16 and 18 have been amended. 

3. Claims 1-14, 16 and 18 are pending. 

Response to Arguments 

4. Rejection of claims 16 and 18 under 35 U.S.C. 1 01 is withdrawn because the 
amendment overcomes the rejection. 

5. Regarding claims 1-14, 16, and 18, applicant's argument has been fully 
considered but not persuasive. 

6. Applicant misunderstood the examiner's proposed combination of Kimata and 
Puri by alleging (pp. 10-1 4) that Puri's motion compensated interpolation has 
disadvantage and therefore it was not obvious to combine Kimata and Puri. The 
interpolation by other GOP's (Chapter 4: inter GOP; Chapter 3 View Generation based 
on Ray-Space) is already disclosed by Kimata. The proposed examiner's modification 
only needs incorporation of idea of conditional encoding (Puri's CMCI), not Puri's motion 
compensated interpolation but interpolation by other GOP's by Kimata. Conditional 
encoding teaches whether the encoding of the current GOP is encoded or not based on 
the magnitude of the interpolation error where the interpolation is done by other GOP's 
only. 
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Claim Rejections - 35 USC §101 

7. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

8. Claims 16 and 18 are rejected under 35 U.S.C. 101 because the claimed 
invention is directed to non-statutory subject matter. 

Claims 16 and 18 recite "A computer-readable storage medium". However, 
computer readable medium includes a transitory medium such as a signal or carrier 
wave. Th applicant is advised to use "A non-transitory computer-readable storage 
medium" instead. 

Claim Rejections - 35 USC §112 

9. The following is a quotation of the first paragraph of 35 U.S.C. 1 1 2: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

10. Claims 1-4, 8-11,12-14, and 16 are rejected under 35 U.S.C. 112, first 
paragraph, as failing to comply with the written description requirement. The claim(s) 
contains subject matter which was not described in the specification in such a way as to 
reasonably convey to one skilled in the relevant art that the inventor(s), at the time the 
application was filed, had possession of the claimed invention. 

Independent claims 1 and 8 recite "an in-GOP image encoding step of encoding 
the image belonging to the relevant GOP when the encoded data of the image is output, 
wherein: when a subject is included in said images belonging to the GOPs and the GOP 
encoding/non-encoding data indicates that the encoded data of the relevant image is 
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not to be decoded, image data of the subject having a viewing position or direction 
which corresponds to the given GOP is generated using data of the images belonging to 
the GOPs other than the given GOP". However, the specification and drawing contradict 
the amended claim. As shown in the drawing (Fig. 2) and the specification (pp.1 3-1 9), 
encoder has two choices largely: In-GOP encoding (110, Fig. 2 ) or non In-GOP 
encoding ( Image Generation Part 114, Fig. 2). In-GOP encoding deals with when the 
encoded data of the image belonging to the given GOP is output. It does not deal with 
"when the relevant image of the given GOP is not encoded". It is non In-GOP encoding 
(114, 115 and 1 1 6, Fig. 2) which does "when a subject is included in said images 
belonging to the GOPs and the GOP encoding/non-encoding data indicates that the 
encoded data of the relevant image is not to be decoded, image data of the subject 
having a viewing position or direction which corresponds to the given GOP is generated 
using data of the images belonging to the GOPs other than the given GOP". Therefore, 
it is a new matter. For the prosecution of the application, examiner interprets that the 
above feature belongs to non In-GOP encoding. 

Independent claim 12 recites "an in-GOP image decoding part in which: .... if the 
GOP encoding/non-encoding data indicates that the encoded data of the relevant image 
is not to be decoded, the in-GOP image decoding part decodes the relevant image by 
using an image generation method which does not use the encoded data of this image, 
wherein: wherein: when a subject is included in said images belonging to the GOPs 
and the GOP encoding/non-encoding data indicates that the encoded data of the 
relevant image is not to be decoded, image data of the subject having a viewing position 
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or direction which corresponds to the given GOP is generated using data of the images 
belonging to the GOPs other than the given GOP". Similar argument as above and 
evident in Fig. 3, it is a contradiction "in-GOP image decoding part decodes the relevant 
image by using an image generation method which does not use the encoded data of 
this image wherein: wherein: when a subject is included in said images belonging to the 
GOPs and the GOP encoding/non-encoding data indicates that the encoded data of the 
relevant image is not to be decoded, image data of the subject having a viewing position 
or direction which corresponds to the given GOP is generated using data of the images 
belonging to the GOPs other than the given GOP". It is a non in-GOP decoding part 
(207, 208 and 209) which "decodes the relevant image by using an image generation 
method which does not use the encoded data of this image wherein: wherein: when a 
subject is included in said images belonging to the GOPs and the GOP encoding/non- 
encoding data indicates that the encoded data of the relevant image is not to be 
decoded, image data of the subject having a viewing position or direction which 
corresponds to the given GOP is generated using data of the images belonging to the 
GOPs other than the given GOP". Therefore, it is a new matter. For the prosecution of 
the application, examiner interprets that the above feature belongs to non In-GOP 
decoding. 

Claims 2-4 and 16 depend on claim 1 . Therefore, they are rejected for the same 
reason too. 

Claims 9-11 depend on claim 8. Therefore, they are rejected for the same 
reason too. 
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Claims 13-14 depend on claim 12. Therefore, they are rejected for the same 
reason too. 



Claim Rejections - 35 USC § 103 

1 1 . The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

12. Claims 1, 3-8, 10-14, 16, and 18 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Kimata (CIT 2004, IEEE) (hereafter referenced as Kimata), in view 
of Puri (Signal Processing Image Communication 2, 1990, pp. 127-1 44) (hereafter 
referenced as Puri). 

Regarding claim 1, Kimata discloses System Design of Free View point Video 
Communication. Kimata specifically discloses A video encoding method (4. Coding 
Method, pp.4- pp.5) that assigns a plurality of images (Fig. 6 -7) to a plurality of GOPs 
(Group of GOP (GoGop), Fig. 6-7) which correspond to different viewing positions or 
directions ( Camera Arrangement Fig. 3 and Fig. 6) ) and encodes images belonging to 
the GOPs as a video image, the method comprising: in-GOP image encoding step of 
encoding the image belonging to the relevant GOP (base GOP and Inter-GOP, Chapter 
4 and Fig. 7). However, Kimata fails to discloses 
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GOP encoding determination step in which: 

if it is determined that each image belonging to a given GOP can be generated on a 
decoding side without using encoded data of a relevant image, it is determined that the 
relevant image is not encoded and no encoded data thereof is output; and 
if it is determined that each image belonging to the given GOP cannot be generated on 
a decoding side unless encoded data of the relevant image is used, then it is 
determined that the relevant image is encoded and the encoded data thereof is output; 
a GOP encoding/non-encoding data encoding step of encoding GOP encoding/non- 
encoding data for indicating whether encoded data of the image belonging to the 
relevant GOP is output, and an in-GOP image encoding step of encoding the image 
belonging to the relevant GOP when the encoded data of the image is output, wherein: 
when a subject is included in said images belonging to the GOPs and the GOP 
encoding/non-encoding data indicates that the encoded data of the relevant image is 
not to be decoded, image data of the subject having a viewing position or direction 
which corresponds to the given GOP is generated using data of the images belonging to 
the GOPs other than the given GOP. 

In the analogous field of endeavor, Puri discloses Video Coding with Motion - 
Compensated Interpolation for CD-ROM Application. Puri specifically discloses 
conditional motion compensated interpolation (CMCI) which is interpolating the skipped 
frames using motion compensated interpolation based on other reference frames, but 
encoding the interpolation error only if it is significant (4. Motion-Compensated 
Interpolation, pp.1 30-1 31), in order to keep the additional information as small as 
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possible (pp.131 , left col, line 9-11). So Puri teaches that if it is determined that an 
image can be generated on a decoding side without using encoded data of a relevant 
image (a skipped frame which has not significant interpolation error), it is determined 
that the relevant image is not encoded and no encoded data thereof is output (not 
encoded if prediction error is not significant); and if it is determined that an image 
cannot be generated on a decoding side unless encoded data of the relevant image is 
used, then it is determined that the relevant image is encoded and the encoded data 
thereof is output (interpolation error is encoded if the prediction error is significant). 
Kimata teaches that the interpolation of the image belonging to the relevant GOP is 
done by Ray-Space method (Kimata: Ray-Space Interpolation, Fig. 3 and 4) based on 
other GOP's (Inter-GOP configuration of Fig.7(b)). It was obvious that Puri's selective 
encoding of frame can be applied to a selective encoding of GOP in Kimata since 
Kimata has GOP based structure (base GOP and Inter-GOP, Chapter 4 and Fig. 7). 

Therefore, given this teaching, it would have been obvious to one of ordinary skill 
in the art at the time invention was made to modify Kimata by incorporating selectively 
encoding the each image belonging to the relevant GOP based on the significance of 
interpolation error with transmission of encoding/non-encoding information (1 bit), in 
order to keep the additional information as small as possible. The Kimata coding 
method, incorporating the Puri selective encoding based on the significance of 
interpolation error, has all the features of claim 1 . Kimata and Puri teaches when a 
subject is included in said images belonging to the GOPs and the GOP encoding/non- 
encoding data indicates that the encoded data of the relevant image is not to be 
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decoded (Puri: No interpolation error is encoded and therefor not decoded), image data 
of the subject having a viewing position or direction which corresponds to the given 
GOP is generated using data of the images belonging to the GOPs other than the given 
GOP (Kimata: Inter-GOP configuration of Fig.7(b)). 

Regarding claim 3, the Kimata coding method, incorporating the Puri selective 
encoding based on the significance of interpolation error, further incorporating adaptive 
selection of In-GOP and non-encoding based on the bandwidth limitation, as applied to 
claim 1 , teaches further comprising: a generation reference-GOP encoding step of 
encoding generation reference-GOP designating data for designating one or more other 
GOPs (Kimata: reference GOP index are defined in GOP header, pp.4, left col, last 3 
lines ) which are used for generating the image (Kimata: Ray-Space Interpolation, Fig. 3 
and 4) belonging to the relevant GOP when the encoded data of this image is not 
output (No-encoding shown in above claim 1) . 

Regarding claim 4, the Kimata coding method, incorporating the Puri selective 
encoding based on the significance of interpolation error, further incorporating adaptive 
selection of In-GOP and non-encoding based on the bandwidth limitation, as applied to 
claim 1 , teaches further comprising: 

a generation data encoding step of encoding generation data (Kimata: Ray-Space 
Approach used as View Generation, pp.3, left col.) for designating an image generation 
method which is used for generating the image (3. Free viewpoint viewer, pp.2) 
belonging to the relevant GOP (Fig. 7(b). GoGOP #2 uses only decoded picture of base 
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GOP (GOP of Other Camera's) ) when the encoded data of this image is not output 
(no-encoding shown in above claim 1). 

Regarding claim 5, The Kimata coding method, incorporating the Puri selective 
encoding based on the significance of interpolation error, as applied to claim 1 , 
discloses A video decoding method (Kimata: Fig. 8 Decoder) for decoding encoded data 
generated by assigning a plurality of images (Kimata: Fig. 6 -7) to a plurality of GOPs 
which correspond to different viewing positions or directions ( Fig. 7 ) and encoding 
images belonging to the GOPs (Kimata: Group of GOP (GoGop), Fig. 6-7) as a video 
image, the method comprising: 

a GOP encoding/non-encoding data decoding step (anticipated by the combination, 
because there is a bit indication encoding-non-encoding as shown above claim 1) of 
decoding GOP encoding/non-encoding data for indicating whether the encoded data of 
each image belonging to each GOP is to be decoded (definition of encoding/non- 
encoding bit); and an in-GOP image decoding step (Fig. 8 includes in-GOP as well as 
inter-GOP) in which if the GOP encoding/non-encoding data indicates that the encoded 
data of a relevant image is to be decoded, the relevant image is decoded by decoding 
the encoded data (Kimata:Fig.8 Decoder); and 

if the GOP encoding/non-encoding data indicates that the encoded data of the relevant 
image is not to be decoded (Puri: encoding the interpolation error only if it is significant, 
pp.1 30-1 31), the relevant image is decoded by using an image generation method 
(Kimata: Ray-Space Approach used as View Generation, pp.3, left col.) ) which does 
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not use the encoded data of this image (does not use image taken by the concerned 
camera, but uses images taken by other cameras, Fig. 7), wherein: 

when a subject is included in said images belonging to the GOPs and the GOP 
encoding/non-encoding data indicates that the encoded data of the relevant image is 
not to be decoded (Puri: No interpolation error is encoded, and therefor not decoded), 
image data of the subject having a viewing position or direction which corresponds to 
the given GOP is generated using data of the images belonging to the GOPs other than 
the given GOP (Kimata: Inter-GOP configuration of Fig. 7(b)). 

Regarding claim 6, the Kimata coding method, incorporating the Puri selective 
encoding based on the significance of interpolation error, as applied to claim 1 , teaches 
further comprising: 

a generation reference-GOP decoding step of decoding generation (Kimata: Reference 
Picture Determination, Fig. 8) reference-GOP designating data for designating one or 
more other GOPs (Kimata: reference GOP index are defined in GOP header, pp.4, left 
col, last 3 lines ) which are used for generating the image (Kimata: Ray-Space 
Interpolation, Fig. 3 and 4) belonging to the relevant GOP when the encoded data of this 
image is not decoded (No-encoding shown in above claim 1, Kimata: Fig. 7b shows 
reference pictures only with other cameras). 

Regarding claim 7, the Kimata coding method, incorporating the Puri selective 
encoding based on the significance of interpolation error, as applied to claim 1 , teaches 
further comprising: 
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a generation data decoding step of decoding generation data for designating an 
image generation method which is used for generating the image (Kimata: Ray-Space 
Interpolation, Fig. 3 and 4) belonging to the relevant GOP(Kimata: C2 images in Fig 
7(b)) when the encoded data of this image is not decoded (no-encoding shown in above 
claim 1). 

Regarding claim 8, the claimed invention is an apparatus claim corresponding to 
the method claim 1 . Therefore, it is rejected for the same reason as claim 1 . 

Regarding claim 10, the claimed invention is an apparatus claim corresponding 
to the method claim 3. Therefore, it is rejected for the same reason as claim 3. 

Regarding claim 11, the claimed invention is an apparatus claim corresponding 
to the method claim 4. Therefore, it is rejected for the same reason as claim 4. 

Regarding claim 12, the claimed invention is an apparatus claim corresponding 
to the method claim 5. Therefore, it is rejected for the same reason as claim 5. 

Regarding claim 13, the claimed invention is an apparatus claim corresponding 
to the method claim 6. Therefore, it is rejected for the same reason as claim 6. 

Regarding claim 14, the claimed invention is an apparatus claim corresponding 
to the method claim 7. Therefore, it is rejected for the same reason as claim 7. 

Regarding claim 16, the claimed invention is a computer-readable medium claim 
corresponding to the method claim 1 . Therefore, it is rejected for the same reason as 
claim 1. 

Regarding claim 18, the claimed invention is a computer-readable medium claim 
corresponding to the method claim 5. Therefore, it is rejected for the same reason as 
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claim 5. 
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1 3. Claims 2 and 9 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Kimata in view of Puri , and further in view of Eifrig (US 5,991 ,447) (hereafter 
referenced as Eifrig). 

Regarding claim 2, the Kimata coding method, incorporating the Puri selective 
encoding based on the significance of interpolation error, as applied to claim 1 , teaches 
image generated by using one or more other GOPs without decoding the encoded data 
of the relevant GOP (no-encoding, but interpolated by other GOP's using Ray-Space), 
and encoding the relevant GOP. 

However, the combination fails to teach the GOP encoding determination step includes 
determining whether an image generated by using one or more other GOPs without 
decoding the encoded data of the relevant GOP is closer to an original image of the 
relevant image in comparison with an image obtained by decoding the 
encoded data, so as to determine whether the image belonging to the relevant 
GOP is to be encoded. 

In analogous field of endeavor, Eifrig discloses Prediction and Coding of Bi- 
Directionally Predicted Video Object Planes for Interlaced Video. Eifrig specifically 
discloses that prediction mode is decided based on SAD (sum of absolute difference 
between prediction and the current picture) {closeness to the original picture) 
calculation of forward, backward, and averaged mode, in order to do efficient coding 
(col. 14, line 16-24). 
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Therefore, given this teaching, it would have been obvious to one of ordinary skill 
in the art at the time invention was made to modify Kimata and Puri by incorporating the 
mode decision based on minimum SAD between In-GOP encoding and non-encoding of 
the relevant image, in order to do efficient coding. The Kimata coding method, 
incorporating the Puri selective encoding based on the significance of interpolation 
error, incorporating the mode decision based on minimum SAD between In-GOP 
encoding and no-encoding of the relevant image, has all the features of claim 2. 

Regarding claim 9, the claimed invention is an apparatus claim corresponding to 
the method claim 2. Therefore, it is rejected for the same reason as claim 2. 



Conclusion 

14. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .1 36(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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1 5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to HEE-YONG KIM whose telephone number is (571)270- 
3669. The examiner can normally be reached on Monday-Thursday, 8:00am-5pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Christopher Kelley can be reached on 571-272-7331 . The fax phone 
number for the organization where this application or proceeding is assigned is 571 - 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/HEE-YONG KIM/ 
Examiner, Art Unit 2482 



/CHRISTOPHER S KELLEY/ 
Supervisory Patent Examiner, Art 
Unit 2482 



